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A SYSTEM AND METHOD FOR AUTHORING, DISTRIBUTING AND 
REPLAYING DERIVATIVE HYPERMEDIA CONTENT 

The present invention relates to the field of information processing on a network 
such as the World Wide Web (WWW) and, more specifically, to authoring, 
5 distributing, and replaying derivative hypermedia content. 

As the World Wide Web (WWW) becomes a more significant information source, 
there is an increasing need to share and reuse WWW information. 

It is herein recognized as being desirable that a WWW user should be enabled to 
place annotations on WWW documents. There are prior art systems which allow 

10 certain kinds of annotations to be placed on WWW documents. The well-known 

Mosaic browser allows annotation of static text to a WWW document. However, 
this system has the limitation that the annotation was stored along with the 
document, and thus required the permission of the document owner to create an 
annotation. This prevented annotation on arbitrary pages as permission was 

15 typically not forthcoming, and it also resulted in security risks for the document 
owners who did grant permission. It also prevented users from having private 
annotations on a document. 

Other systems, such as the ViewDirector Prism system, from TMS Sequoia, and 
the HotOffTheWeb system from Insight Development, allow graphical annotation 

20 on WWW documents. However, these annotations are static in nature, meaning 
that there is no timing or navigation information stored with them. Such a system 
cannot easily be used to author a tour through several pages as is herein recognized 
to be desirable. Moreover, these systems do not support voice annotation. 
A natural mechanism for explanation or exposition is the so-called dynamic 

25 annotation, such as is described for image data in U.S. Patent No. 5,838,313, issued 

17-Nov-1998 in the names of Hou, Tai-Yuan et al. (Siemens Docket #95P7547), 
whereof the disclosure is hereby incorporated by reference to the extent it is not 
inconsistent with the present application. Briefly, that application discloses a 
multimedia-based reporting system which utilizes a command interpreter to receive 

30 inputs from a user and forwards the inputs to an open report handler, a save report 
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handler, a delete current report handler, a mail report handler, a static annotation 
handler, a dynamic annotation handler including a recording and playback handler, 
an annotation on annotation handler, an attach media handler and a print report 
handler. Most relevantly to the current invention, the dynamic annotation handler 
allows the individual to record/playback annotations from the user input. 

Creating an annotation involves recording the mouse gestures, drawings, and voice 
of the annotation author, along with timing information, so that the annotation can 
be later replayed in just the way that the author created it. Unlike the Mosaic, 
Prism, and Sequoia systems mentioned above, users of a dynamic annotation 
system could see the graphical annotations unfold in time with the comments of the 
author. 

In accordance with an aspect of the present invention, there is described a tool 
which allows a WWW user to author a new kind of WWW content which is 
derivative from existing WWW documents. A user is able to add annotations to 
WWW documents, and record a "guided tour" through a series of WWW 
documents. The resulting tour is a combination of the existing WWW documents, 
the ordering and timing introduced by the author, and the author's annotations. 

In accordance with another aspect of the invention, the idea of dynamic annotation 
is extended to also record navigation events. In this way, hypermedia documents 
can be toured in an annotation. 

In accordance with an aspect of the present invention, there is described a 
multimedia based system for authoring, distributing, and replaying derivative 
hypermedia content, including a tool which, for example, allows a WWW user to 
author a new kind of WWW content which is derivative from existing WWW 
documents. 

In accordance with another aspect of the invention, there are primarily three 
components to the system: an authoring system for recording dynamic annotations 
on hypermedia, a distribution system for distribution of those annotations, and a 
playing system for playing those annotations. The authoring and playing systems 
are comprised in a hypermedia browser, while the distribution system is partly 



comprised in the browser and partly in a special remote server. The part of the 
distribution system within the browser is called the annotation manager, and the 
remote server is called the annotation server. 

The present invention, as hereinafter described by way of exemplary embodiments, 
5 integrates the idea of dynamic annotation with a hypermedia browser such as a 
WWW browser. A traditional hypermedia browser is augmented with systems for 
recording, distributing, and playing dynamic annotations of hypermedia 
documents. The invention generates new opportunities for creating and sharing 
derivative content from hypermedia sources such as the WWW. 

[0 The invention will be more fully understood from the following detailed 
description of the preferred embodiments, in conjunction with the drawing, in 
which 

Figure 1 shows the overall architecture of the system, as implemented for use with 
the WWW; 

15 Figure 2 shows a layout of an embodiment in accordance with the present 
invention; Figure 3 shows the flow of control of a hypermedia browser in 
accordance with the present invention; 

Figure 4 shows the operation of the system in accordance with the present 
invention in annotation mode; 
20 Figure 5, in reference to a system in accordance with the invention, shows the 
operation of the annotation distribution system for the storing of an annotation. 

Figure 6, in reference to a system in accordance with the invention, shows the 
operation of the annotation distribution system for the alerting users of the presence 
of annotations. 

25 Figure 7 shows the operation of the system in accordance with the present 
invention, during annotation playback. 

As the most widely used form of hypermedia is the WWW, the invention will be 
illustratively described herein in terms of the WWW, although the concepts and 
general design are fully applicable to any hypermedia system. 
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Figure 1 shows the overall architecture of the system, as implemented for use with 
the WWW. 

A traditional WWW browser has several components, notably a network interface 
which can access documents from remote sites on the network using well-known 
5 protocols such as HTTP, and a document renderer which converts structured 
documents into a form which is understandable by a user. These components are 
widely available for incorporation into extended systems, as for example from the 
FastNet ActiveX component set from NetMasters LLC. To allow the users to 
access the new features of the current invention, several new user-interface objects 
10 are added. Each of these objects is described with the relevant new component. 

Figure 2 shows a layout of a typical instance of this invention, with the new user 
interface objects indicated. 

The invention described in the afore-mentioned U.S. Patent No. 5,838,313 allows 
the user to author dynamic annotations on electronic documents. The present 
15 invention allows the user to author annotations within a program for browsing 

hypermedia and furthermore exhibits three novel features: the capture and 
recording of navigation events, the capability to include multiple documents within 
an annotation, and the capability to insert hyperlinks to existing annotations. 

On pages 8 and 9 of the afore-mentioned U.S. Patent No. 5,838,313, a list is 
20 provided of "significant events" which are to be recorded in the annotation. 

Briefly, it is therein stated, in the context of events recorded during recording that 
two types of discrete events are included, the user-activated events and the time- 
sliced events. The user-activated events, of concern herein, have the format 
(event__name, time, parameters). Events were described therein for many typical 
25 user-activated events such as moving a user-interface object on the screen, 

changing the color of some text, erasing a graphic, etc. 

In accordance with the described embodiment of the present invention, the list of 
significant events is extended to include the following navigation events, which are 
common in hypermedia browsers: 

30 - following a hyperlink 
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- invoking "forward", "back", "refresh", or "home" browser commands 

- jumping to a document through the use of a history list 

- jumping to a document by invoking a "bookmark" 

- scrolling the document in the viewable area 

5 As with other events described in the afore-mentioned U.S. Patent No. 5,838,313, 
in accordance with the described embodiment of the present invention, each event 
is stored as (event_name, time, parameters). The first four navigation events are all 
stored with the event_name NEW_DOCUMENT, and with a single parameter that 
stores the target document ID. Normally, this ID uniquely identifies the document. 

10 In the case of WWW documents, the Universal Resource Locator (URL) serves as 
the ID. The scrolling events are stored as with the event name SCROLL and have 
one parameter to indicate the scrolling direction (left, right, up or down), and a 
second parameter to indicate the magnitude of the scroll. 

Figure 4 shows the operation of the system in accordance with the present 
15 invention in annotation mode. During the authoring of an annotation, the 

occurrence of a navigation event other than a SCROLL event causes the 
hypermedia browser to load and render a new document in the usual way. The 
recording of the annotation, including the audio recording and the clock, pauses 
while the new document is loading, and resumes when the loading is completed. 
20 This allows multiple documents to be annotated within the same annotation. As 
the clock is paused during loading, times for all events are stored independent of 
the loading time of the various documents accessed. For example, if the user 
causes a MOUSE_MOVE event, the timestamp for this event would be the number 
of milliseconds since the annotation began, minus any time spent downloading. 

25 Another hovel feature of the dynamic annotation authoring component is the 

capability to create hyperlinks to existing annotations. The user creates a hyperlink 
by clicking on the target annotation identifier (marked H in Figure 2) in the 
annotation manager box (marked G in Figure 2) both of which are described in the 
next section. The hyperlink is stored as an event with event_name 



5 



ANN OT ATIONJLINK, a time, and one parameter giving the identifier of the 
target annotation. 

In order to allow the user to access the features of the dynamic annotation 
authoring system from within the hypermedia browser, two user-interface objects 
5 were added to the traditional browser interface. These are: 

- a browser-mode box with three buttons: browse, annotate and watch (A,B, and 
C in Figure 2) 

- an annotation-mode box with two buttons: scribble and text (D and E in Figure 
2) 

10 A mode box is a collection of options, only one of which can be valid at a time. 
Figure 3 shows the flow of control of the hypermedia browser as determined by the 
two new mode boxes. The browser-mode is normally Browse, in which case the 
annotation-mode box is disabled (shown). In Browse mode, the normal WWW 
operations are in effect. Clicking on the annotate button switches the browser- 

15 mode to Annotate. In this mode (see Figure 4), events (including the navigation 
events listed in the previous section) are captured and stored as in the afore- 
mentioned U.S. Patent No. 5,838,313, depending on the annotation-mode. The 
annotation-mode box is enabled. Clicking on the text button sets the annotation 
mode to Text, and clicking on the scribble button sets the annotation-mode to 

20 Scribble. In both annotation-modes, mouse events are captured. In the Scribble 
mode, MOUSE_MOVE events following a MOUSE_DOWN event, but preceding 
the next MOUSE_UP event cause a line to be drawn. 

In Text mode, no lines are drawn, and MOUSE_DOWN events are used to fix the 
X,Y coordinates for TYPE events. The first TYPE event receives the X,Y 
25 coordinates of the last MOUSE_DOWN event, and successive TYPE events 
receive the same Y coordinate and an incremented X coordinate appropriate for the 
font size. 

As shown in Figure 3, clicking on the browse button returns the browser to Browse 
mode, and ends the annotation. Clicking on the watch button switches the browser 
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to Watch mode in which the current annotation is replayed, as described in section 
"A Means for Playing Dynamic Annotations of Hypermedia " 

A second principal component of the invention is a system for distributing 
annotations. This system includes elements within the hypermedia browser, called 
5 the annotation manager, and elements at a remote server, called the annotation 
server. In general, the system allows the user to store annotations at the annotation 
server. When another user accesses the same hypermedia document, the browser 
checks the server for any annotations on that document. If there are, these 
annotations are made available to the new user for viewing. 

10 In order to allow the user to access the features of the dynamic annotation 

distribution system from within the hypermedia browser two user-interface objects 
were added to the traditional browser interface: 

- a save button (labeled F in Figure 2) 

- an annotation manager box (labeled G in Figure 2), which contains annotation 
15 identifiers (labeled H in Figure 2). 

After having authored an annotation and having returned to Browse mode, the user 
has the option of storing it on the annotation server. User interface element F is the 
save button. As shown in Figure 5, pressing the save button invokes a dialog box 
of the traditional kind, which allows the user to enter a name for the annotation and 

20 their own name. Once a name is entered, the annotation is written to the local disk 
as an annotation file, as described in the afore-mentioned U.S. Patent No. 
5,838,313. The file is then transferred to the annotation server through the use of 
FTP or a comparable file transfer protocol, along with its name and/or the name of 
the author. An optional feature is that the annotation manager generates a visual 

25 icon representing the annotation, and sends that to the annotation server as well. 

At the annotation server, the annotation file is stored in the annotation store. The 
annotation store can be any kind of traditional database. The annotation file entry 
is indexed by the unique ID of each document included within the annotation. 
These IDs can be extracted directly from the NEWjDOCUMENT events in the 
30 annotation file. 
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As shown in Figure 6, each time a user accesses a document using the hypermedia 
browser, the annotation manager queries the annotation server using the ID of the 
document. If any annotations are stored in the annotation store which have been 
indexed with that ID, the names of these annotations, their authors, and their icons 
5 if available, are returned to the annotation manager. These names and icons are 
then displayed to the user in the annotation manager box, which is user interface 
element G, each separately, as shown in user interface element H. The user is thus 
made aware of any annotations which may be available for this document. 

There are many protocols for restricting a user's rights to viewing annotations, each 
10 of which may be useful in different situations. Such protocols and will not be 
particularly described herein, in that the present invention is compatible with a 
number of possible choices of access protocol. 

While the invention disclosed in the afore-mentioned U.S. Patent No. 5,838,313 
allows the user to play dynamic annotations on electronic documents; the present 

15 invention enables the playing of annotations within a program for browsing 
hypermedia, and extends operation with three new features: the capability to play 
annotations over multiple documents, an enhanced synchronization system to 
handle arbitrary document loading times, and a means for displaying temporal 
hyperlinks to other annotations. 

20 The user of the system can play previously recorded annotations by clicking on any 

of the annotations listed in the annotation manager box described previously. In 
addition, an annotation author can view the annotation he has just made by 
switching the browser into Watch mode, as shown in Figure 3. The operation of 
the system during playback is shown in Figure 7. 

25 As described in the afore-mentioned U.S. Patent No. 5,838,313, annotations are 
played by processing each event that has been stored, in turn, while maintaining the 
timing as much as possible. In the case of NEW_DOCUMENT events, the ID of 
the document is passed to the traditional browser modules for loading and 
rendering. Any subsequent events, such as drawing of lines or mouse gestures, are 
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displayed on the new document. In this way, annotations over multiple documents 
can be played. 

A difficulty posed by multiple-document annotation, as opposed to single 
document annotation, is that the time required to load and render a new document 
is not consistent from recording to playback, or even from one playback to another. 
This is particularly true with WWW documents, for which the loading time can 
vary substantially. At playback time, this can cause the recorded audio and events, 
which are on an absolute timeline, to become unsynchronized with the associated 
document. In the present invention, this problem is solved by pausing both the 
recording and playback of annotations during the loading and rendering of new 
documents. Specifically, when a NEW_DOCUMENT event is processed in Watch 
mode, playback is paused until the browser notifies the playback module that 
loading and rendering are complete. Further, the timestamps of all events are 
stored and interpreted relative to the completion of the loading of the most recent 
document. In this way, arbitrary loading times can be tolerated with no loss of 
synchronization. 

As described previously, the present invention allows annotation authors to create 
hyperlinks to existing annotations. These are stored as ANNOTATION_LINK 
events. During playback, ANNOTATION JLINK events are interpreted by causing 
a special "link button" to appear on the browser temporarily. In the described 
exemplary embodiment, the link button appears initially bright red, and then slowly 
fades to white over a period off seconds. Empirical tests show that five seconds is 
an appropriate value for t. After the link button becomes completely white, it is 
removed. While the link button is visible, the user may click on it. Clicking on the 
link causes the playback of the current annotation to pause, and playing to begin of 
the target annotation, that is, the annotation with the ID stored in the 
ANNOTATIONJJNK event. When the target annotation concludes, the playback 
of the previous annotation resumes at the event where it left off. 

As there may be several annotation hyperlinks followed during a single playback 
session, a stack is required to maintain the state of the playback. Whenever the 
link button is clicked, the current annotation, and the index of the next event in the 



annotation, are placed on the stack. Whenever an annotation concludes, the stack 
is checked to see if there are any outstanding annotations to resume. If the stack is 
empty, the playback is finished; otherwise the annotation on the top of the stack is 
removed, and playback resumes with that annotation. 

While the invention has been described by way of exemplary embodiments, it will 
be understood by one of skill in the art to which it pertains that various changes and 
modifications may be made to the invention without departing from the spirit of the 
invention and that such changes and modifications are intended to be covered by 
the claims following. 
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CLAIMS 



What is claimed is: 



1. A system for authoring, distributing, and replaying derivative hypermedia 
5 content, said system comprising: 

an authoring system for recording dynamic annotations; 

a distribution system for distributing said dynamic annotations which have been 
recorded in the preceding step; and 

a playing system for playing said dynamic annotations which have been distributed 
10 in the preceding step. 

2. A system for authoring, distributing, and replaying derivative hypermedia 
content in accordance with claim 1, wherein said authoring system for recording 
dynamic annotations comprises: 

a system for capturing navigation events. 

15 

3. A system for authoring, distributing, and replaying derivative hypermedia 
content in accordance with claim 2, wherein said playing system for playing said 
dynamic annotations comprises: 

20 a system for playing said navigation events. 



4. A system for authoring, distributing, and replaying derivative hypermedia 
content in accordance with claim 1 , wherein said dynamic annotations are 

25 recorded on hypermedia. 

5. A system for authoring, distributing, and replaying derivative hypermedia 
content in accordance with claim 4, wherein hyperlinks may be created and 
followed between said dynamic annotations. 

11 



6. A system for authoring, distributing, and replaying derivative hypermedia 
content in accordance with claim 5, including a hypermedia browser, said 
authoring and playing systems being included in said hypermedia browser and said 
distribution system being in part comprised in said hypermedia browser. 

5 7. A system for authoring, distributing, and replaying derivative hypermedia 

content in accordance with claim 6, including a remote annotation server, 

8. A system for authoring, distributing, and replaying derivative hypermedia 
content in accordance with claim 7, wherein said distribution system is partly 
comprised within said hypermedia browser and partly within said remote 

10 annotation server. 

9. A system for authoring, distributing, and replaying derivative hypermedia 
content in accordance with claim 8, wherein said remote annotation server further 
comprises an annotation store, and a mechanism for indexing, retrieving and 
transferring annotation files. 

15 10. A system, for use with a communications web or net, said net comprising at 

least one of the Internet and an Intranet, said system being for authoring, 
distributing, and replaying derivative hypermedia content, said system comprising: 

an annotation server for being coupled to said communications net; 

a remote net server for being coupled to said communications web; and 

20 a net browser for being coupled to said communications web. 

11. A system in accordance with claim 10, wherein said net browser comprises: 
a system for capturing navigation events. 

25 
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12. A system in accordance with claim 11, wherein said annotation server 
comprises an annotation store. 

13. A system in accordance with claim 10 , in which hyperlinks may be created 
and followed between annotations. 

5 14. A system in accordance with claim 13, wherein said net browser comprises 

traditional hypermedia browser components, an annotation manager, an annotation 
recorder, and an annotation player. 

15. A system in accordance with claim 10, wherein said system has a selectable 
Browse operating mode wherein said net browser functions in a traditional 

10 hypermedia browser manner, an Annotate mode wherein events generated by a user 
are recorded, and a Watch mode wherein a most recent annotation is played. 

16. A system in accordance with claim 10, wherein said system displays 
available annotations for each document that a user visits. 

17. A method for use with a communications web or net, said net comprising at 
15 least one of the Internet and an Intranet, said method being for authoring, 

distributing, and replaying derivative hypermedia content, wherein said method 
comprises: 

coupling each of at least one remote server, an annotation server, an annotation 
manager, and a net browser to said communications net; and 

20 selecting one of a Browse mode, an Annotate mode, and a Watch mode for 

respectively causing said net browser to function in a traditional hypermedia 
browser manner, causing events generated by a user to be recorded, and for causing 
a most recent annotation to be displayed. 

18. A method in accordance with claim 17, wherein said causing events 
25 generated by a user to be recorded includes causing navigation events to be 

recorded. 

13 



18. A method in accordance with claim 17, wherein said causing a most 
recent annotation to be displayed includes causing navigation events to be 
displayed. 



5 19. A method in accordance with claim 17, wherein said Browse mode, said 

Annotate mode, and said Watch mode are selectable by screen button functions. 

20. A method in accordance with claim 17, comprising the steps of, when said 
Annotate mode is selected for recording events: 

selecting one of a drawing mode and a typing mode. 

10 21 . A method in accordance with claim 17, comprising the step of, upon 
selection of an icon or text representing an annotation: 

generating a link annotation event using the annotation ID of said item selected. 

22. A method in accordance with claim 17, comprising the steps of, when said 
Annotate mode is selected for recording events, and upon one of (a) selection of an 
15 item in an Annotation box, and (b) generation of a non-scroll navigation event: 

pausing recording of annotation; 

loading a document with given ID; 

rendering said document by said net browser; and 

resuming recording of annotation. 



20 23. A method in accordance with claim 17, comprising the steps of, when said 
Browse mode is selected and upon activation of a save function: 

entering information including a user-selected name associated with a most 
recent annotation; 

generating an annotation file; 

25 sending said name and said annotation file to said annotation server; 
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extracting document ID's from NEWJDOCUMENT events in said 
annotation; and 

storing a file with said information in an Annotation Store indexed by said 
document ID's. 

24. A method in accordance with claim 22, comprising the steps of, when said 
Browse mode is selected and upon execution of a non-scroll navigation event 
which causes a new document to be loaded by the browser: 

said Annotation manager sending an ID of said new document to said 
Annotation server; upon said Annotation server finding any annotations stored for 
said ID of a new document, said Annotation server sending annotation files along 
with any associated names and icons to said Annotation Manager, said Annotation 
Manager displaying annotation's name, author's name if any, and icon if any, of 
each annotation in an Annotation Manager box. 

25. A method in accordance with claim 24, comprising the steps of, when said 
Browse mode is selected, and upon execution of a non-scroll navigation event 
which causes a new document to be loaded by the browser: 

said Annotation manager sending an ID of a new document to said Annotation 
server; 

upon said Annotation server finding no annotations stored for said ID of a new 
document, said Annotation alerting said Annotation Manager that there are no 
annotations for a current page. 

27. A method in accordance with claim 17, comprising the steps of, when said 
Watch mode is selected: 

processing events in a traditional manner in a Main Annotation Playback 

Loop. 

15 



28. A method in accordance with claim 27, comprising the steps of, when said 
Watch mode is selected, and upon processing of a non-scroll navigation event, 

pausing playback of annotation; 

loading a document having a selected ID; 

5 rendering said document; and 

continuing playback of said annotation. 

29, A method in accordance with claim 28, comprising the steps of: 

if said playback is complete, verifying whether an Annotation Stack is empty and, 
if so, 

10 switching to said Browse mode and, if not, taking the top annotation from said 

stack and beginning playback from the designated event. 



30. A method in accordance with claim 29, comprising the steps of, when said 
Watch mode is selected, and upon processing of a link annotation event, 

15 creating a temporary Link Button which remains available to the user for a 

pre-determined length of time. 

31. A method in accordance with claim 30, comprising the steps of, when said 
Watch mode is selected, and upon activating said Link Button, 

placing the ID of the active annotation, along with the index of the next 
20 event in the active annotation on the top of the stack; 

beginning playback of an annotation referenced in the most recent link 
annotation event. 



25 



32. A method in accordance with claim 31, comprising the steps of: 
if said playback is complete, 

16 



verifying whether an Annotation Stack is empty and, if so, 
switching to said Browse mode and, if not, taking a top annotation from said stack 
and beginning playback from a designated event. 
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ABSTRACT OF THE DISCLOSURE 



A system for authoring, distributing, and replaying derivative hypermedia content 
comprises an authoring system for recording dynamic annotations; a distribution 
system for distributing the dynamic annotations which have been recorded in the 
preceding step; and a playing system for playing the dynamic annotations which 
have been distributed in the preceding step. The system is applicable to the 
Internet or World-Wide Web (WWW) and/or to Intranet use. 
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The Annotation Server 
extracts the document 

IDs from the 
NEW„DOCUMENT 
events in the annotation. 



J 

The Annotation Server 
stores the file and 
information in the 
Annotation Store indexed 
by the document IDs. 



User causes Hypermedia Browser 
to load a new document 
while in Browser mode. 



Annotation Manager 
sends ID of new document 
to Annotation Server 
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Annotation Server looks 
for any annotation files 
indexed under for this 
document ID in the 
annotation store. 



/ Are any \. 
/ annotations yes 
\ stored for this y~— — ► 
\. document ID / 


Annotation server sends 
annotation files along with 
any associated names and 
icons to the Annotation Manager 


[No 




r 


Annotation Server alerts 
Annotation Manager that 
there are no annotations 
for this page. 




Annotation Manager displays 
annotations name, authors name, 
and/or icon of each annotation in 
the annotation manager box. 



Control is returned to the 
Hypermedia Browser' s 
traditional components 



Watch mode is entered, 
or annotation in annotation 
manager box is clicked 



FIG. 7. 





Main Annotation 
Playback Loop 

Events are processed 
as in the prior art. 


LINK_ANNOTATION 
event processed 


Link button is 
displayed. It will 
remain visible for t 
seconds. 


1 ► 


< 1 





NEW_DOCUMENT/ 
event processed 



Playback of 
annotation 
paused 



Document with given 
ID is loaded and 
rendered. 




Place the active 
annotation and the 
index of next event 
on to the stack. 



Take top annotation 
from annotation stack 
and begin playback 
from the designated event. 



Playback of the 
annotation 
continues 



Begin playback of the 
annotation referenced 

in the most recent 
LINK.ANNOTATION 
event. 
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DECLARATION FOR PATENT APPLICATION & POWER OF ATTORNEY 



As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below 
next to my name, 

I believe we are the original, first and sole inventor {if only one 
name is listed below) or an original, first and joint inventor {if plural 
names are listed below) of the subject matter which is claimed and for 
which a patent is sought on the invention entitled: 

A SYSTEM AND METHOD FOR AUTHORING , DISTRIBUTING 
AND REPLAYING DERIVATIVE HYPERMEDIA CONTENT 



the specification of which 
X is attached hereto, and 

was filed on as Application Serial No. 

and was amended on (if applicable) 

Listing of named inventor(s): Michael WYNBLATT; Arturo PIZANO; Arding HSU ; 

Daniel C. BENSON; Chellury R. SASTRY; Vivek 
AGRAWALA 

I hereby state that I have reviewed and understand the contents of 
the above identified specification, including the claims, as amended by any 
amendment referred to above. 

1 acknowledge the duty to disclose all information known to me to be 
material to patentability as defined in Title 37, Code of Federal 
Regulations § 1.56. 

I hereby claim foreign priority benefits under Title 35, United 
States Codes, § 119 of any foreign application ( s ) for patent or inventor's 
certificate listed below and have also identified below any foreign 
application for patent or inventor's certificate having a filing date 
before that of the application on which priority is claimed: 



PRIOR FOREIGN 


APPLICATION (S) 




Priority 


claimed 




(Number) 


(Country) 


( Day/month/year 


filed) 


Yes 


No 


(Number) 


(Country) 


{Day/month/year 


filed) 


Yes 


No 


(Number) 


(Country) 


(Day/month/year 


filed) 


Yes 


No 



I hereby claim the benefits under Title 35, United States Code, § 120 
of any United States application (s ) listed below and, insofar as the 
subject matter of each of the claims of this application is not disclosed 
in the prior United States application in the manner provided by the first 
paragraph of Title 35, United States Code, § 112, we acknowledge the duty 
to disclose material information as defined in Title 37, Code of Federal 
Regulations, § 1.56(a) which occurred between the filing date of the prior 
application and the national or PCT international filing date of this 
application : 

(Application Serial No.) (Filing date) (Status) 

(patented, pending, abandoned) 



(Application Serial No.) 



(Filing date) (Status) 

(patented, pending, abandoned) 
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Power of Attorney : As a named inventor, I hereby appoint the following 
attorney (s) and/or agent (s) to prosecute this application and transact all 
business in the Patent and Trademark Office connected therewith. (list 
name and registration number) 

Adel A. Ahmed, Reg. No. 29,606; Stanton C. Braden, Reg. No. 32,556; Robert 
T. Canavan, Reg. No. 37,592; Dexter K. Chin, Reg. No. 38,842; Joseph S. 
Codispoti, Reg. No. 31,819; Aaron C. Deditch, Reg. No. 33,865; Lawrence C. 
Edelman, Reg. No. 29,299; Mark H. Jay, Reg. No. 27,507; Paul Kiel, Reg. No. 
40,677; Rosa S. Kim, Reg. No. 39,728; David W. Laub, Reg. No. 38,708; Peter 
A. Luccarelli, Jr., Reg. No. 29,750; Jeffrey P. Morris, Reg. No. 25,307; 
Donald B. Paschburg, Reg. No. 33,753; Jeff Slusher, Reg. No. 34,729; Darryl 
A. Smith, Reg. No. 37,756; Heather S. Vance, Reg. No. 39,033; Scott T. 
Weingaertner, Reg. No. 37,756; Robert A. Whitman, Reg. No. 36,966; Ira Lee 
Zebrak, Reg. No. 31,147 

Send correspondence to : 

Siemens Corporation 
Intellectual Property Department 
18 6 Wood Avenue South 
Iselin, NJ 08830 

Direct telephone calls to : 

Elsa Keller, Legal Administrator (732) 321-3026 

I hereby declare that all statements made herein on my own knowledge 
are true and that all statements made on information and belief are 
believed to be true, and further that these statements were made with the 
knowledge that willful false statements and the like so made are punishable 
by^ fine or imprisonment, or both, under Section 1001 of Title 18 of the 
United States Code and that such willful false statements may jeopardize 
the validity of the application or any patent issuing thereon. 



Full name of sole 

or first inventor Michael WYNBLATT 



Inventor ' s signature 




Date 

Residence Robbinsville, New Jersey 

Citizenship U.S. 

Post Office Address 103 Endsleigh Court, Robbinsville, NJ 08691 



Full name of 
second inventor Arturo PI&ANO 



Inventor's signature 
Date 11 1\&TSJ§ 




Residence Belle Mead, New Jersey 
Citizenship Mexico 



Post Office Address 34 Ketcham Road, Belle Mead, NJ 08502 
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Full name of 

third inventor Arding HSU 



Inventor's signature ^ijf^^? 



Date n / /<P/?P 





Re s iden ce Kendall Park, New Jersey 
Citizenship U.S. 



Post Office Address 4 Musket Drive, Kendall Park, NJ 08 824 



Full name of 

fourth inventor Daniel C. BENSON 



Inventor 1 s signature^ 
Date 



Residence Seattle, Washington 
Citizenship U.S. 



Post Office Address 3510 NE 182 nd , Seattle, WA 98155 



Full name of 

fifth inventor Chellury R. SASTRY 



Inventor 1 s signature ^ ^ u^^^w ,_ST^d ~h ^ ~Pf" 

Date ) > J I & I ^ §T 

Residence jTo o X F^X £ w h'tyt^ N^/nsio"*?, ^ CT Of A" J 6 
Citizenship India 

Post Office Address S e cl ^ C'^b t \f 



Full name of 

sixth inventor Vivek AGRAWALA 



Inventor 1 s signature_ 
Date 



Re s idence Plainsboro, New Jersey 
Citizenship India 



Post Office Address 18-05 Fox Run Drive, Plainsboro, NJ 08536 



